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0.  SUMMARY 


0.1  Report  Period 

This  period  of  contract  activity  primarily  involved  software 
development  and  system  implementation  of  the  newly  developed  configuration 
of  information  selection  and  pacing  models  in  the  multi-man  situation. 

In  addition,  the  configuration  of  models  previously  developed  for  the 
single-man  system  was  refined,  demonstrated,  and  documented.  The  specific 
tasks  accomplished  during  the  report  period  included: 

Multi -Man  System 

(1)  The  program  supervisor  and  subject  processes  were  coded, 
tested,  and  implemented  on  the  POP  11/45  computer  system. 

(2)  The  display  software  for  monitoring  and  controlling  operator 
stations  was  designed  and  coded. 

(3)  The  interprocess  communication  software  was  debugged  with 
respect  to  the  UNIX  operating  system. 

(4)  The  message  base  for  the  Tactical  and  Negotiations  Game  (TNG) 
scenario  was  extended  and  adapted  for  interactive  play  on 
the  computer  system. 

Single-Man  System 

(1)  The  pacing  component  of  the  supervisory  system  for  managing 
information  flow  was  further  refined  and  subjected  to  an 
experimental  demonstration  within  the  context  of  the  TNG 
simulation. 
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(2)  A  description  of  the  integrated  configuration  of  the 

information  selection  and  pacing  models  -  including  design, 
implementation,  operation,  and  demonstration  -  was  prepared 
and  appears  as  the  text  of  Sections  1  through  5  of  this 
report. 

0.2  Next  Period 

The  next  period  of  contract  activity  will  concentrate  on  the 
refinement  and  completion  of  the  software  for  the  multi -man  system  and  the 
experimental  demonstration  of  system  operation.  The  specific  items  of  work 
include  the  following: 

(1)  Test  and  refine  model  components  and  their  configuration. 

(2)  Improve  operator  display  format  according  to  human  factors 
guidelines. 

(3)  Implement  full  message  complement  within  TNG  data  base. 

(4)  Code,  implement,  and  debug  software  for  on-line  monitoring  and 
storage  of  operator  performance  measures. 

(5)  Conduct  simulations  and  manual  run-throughs  in  order  to 
specify  initial  parameters  and  procedures  related  to  the 
C3  task. 

(6)  Demonstrate  and  evaluate  system  performance. 


PROGRAM  TASK 
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TABLE  0-1.  PROGRAM  MILESTONES 


1.  INTRODUCTION 


Technical  advances  have  enhanced  the  capability  of  military 
operations  to  the  extent  that  the  amount  of  relevant  information,  and 
the  rate  at  which  it  is  acquired,  have  greatly  increased.  For  commanders 
to  make  tactical  decisions  responsive  to  the  rapidly  changing  succession 
of  events  requires  information  to  be  processed  more  efficiently  and  more 
effectively  than  ever  before.  To  meet  this  need,  new  computer-based 
command,  control,  and  communication  (C3)  systems  are  being  developed  and 
implemented.  These  systems  will  aid  in  the  collection,  processing  (e.g., 
storage  and  communication,  analysis,  and  interpretation),  and  utilization 
of  different  types  and  amounts  of  military  data.  The  overall  process  is 
cyclic  --  as  information  is  being  used,  other  information  is  being 
processed,  and  new  information  is  being  sought  and  collected.  The 
dynamics  of  information  flow  are,  therefore,  of  critical  importance  and 
must  be  constantly  monitored  and  directed. 

The  consensus  concerning  current  computer-based  military  systems 
for  C3  operations  is  that  they  have  increased  the  density  of  information 
flow  to  such  an  extent  as  to  overwhelm  a  commander  and  his  staff. 

Research  is  therefore  necessary  to  determine  how  to  control  information 
flow  so  as  to  best  match  the  machine  capability  with  the  human  function 
in  the  man-computer  interaction.  In  particular,  the  programmable 
features  of  computer  systems  should  be  exploited  so  that  the 
behavioral  dimensions  of  information  flow,  such  as  message  selection, 
routing,  sequencing,  pacing,  etc.,  can  be  monitored  and  maintained  in 
a  mix  optimal  for  command  decision  making. 

Future  C3  systems  will  be  characterized  by  an  increasing  emphasis 
on  the  man/computer  interaction.  Experience  and  experiment  have  shown 
that  the  most  cost-effective  computer-based  information  systems  are  those 


which  most  closely  match  the  requirements  of  their  users.  Accordingly, 
a  major  goal  of  C3  system  design  will  be  to  provide  individualized 
organization  and  management  of  dynamic  information  flow. 


The  purpose  of  the  research  undertaken  here  is  to  develop  and 
demonstrate  means  by  which  computer-based  models  of  the  individual  user 
can  be  employed  to  provide  the  critical  function  of  information  control . 
The  goal  is  to  allow  each  user  consistently  to  obtain  information 
that  is  both  relevant  and  timely  with  regard  to  his  individual  processing 
characteristics  and  immediate  decision  making  needs.  Such  an  aid  could 
improve  system  effectiveness  by  increasing  the  efficiency  of  information 
selection  and  presentation. 


The  specific  objective  of  the  research  program  was  to  develop, 
implement,  and  demonstrate  a  prototype  adaptive  system  for  automatically 
selecting  and  pacing  information  messages  for  an  individual  user 
performing  a  complex  information  processing  task.  The  design  of  the 
system  is  based  on  the  logical  integration  of  separate  models  for 
information  selection  and  information  pacing.  Subsequent  sections  of 
this  paper  present  the  components  and  dynamics  of  the  system,  describe 
an  experimental  demonstration  of  its  operation,  and  discuss  extended 


2.  SYSTEM  DESCRIPTION 


The  domain  of  application  for  the  information  selection  and 
pacing  models  developed  here  is  a  dynamic  local  environment,  where  new 
information  of  the  same  general  type  must  be  processed  repeatedly.  Such 
environments  are  common  in  modern  computer-based  C3  operations.  If  the 
human  operator  and  the  computer  can  be  considered  as  representing  a 
single  man-machine  system,  then  the  goal  of  the  technique  is  to  provide 
the  operator  with  information  which  will  improve  the  overall  decision 
output  of  the  system.  The  two  models  composing  the  system  are  described 
separately  below. 

2.1  Information  Selection  Model 


The  basic  concept  of  the  information  selection  model  is  illustrated 
in  Figure  2-1.  The  message  universe  includes  all  information  currently 
available  to  the  operator  or  system  user.  In  the  manual  mode,  the 
recipient  continuously  selects  messages  in  accord  with  a  selection 
strategy.  A  strategy  represents  individual  preference  for  information 
in  response  to  situational  needs.  In  the  automatic  mode,  an  adaptive 
program  automatically  selects  messages  for  the  user  on  the  basis  of  the 
individual's  selection  strategy  which  the  program  has  learned. 

The  factors  which  characterize  an  individual's  strategy  are 
incorporated  in  an  adaptive  multi -attribute  utility  model.  In  this 
model,  messages  are  decomposed  into  measurable  attributes  such  as  content 
area,  level  of  specificity  (summary  or  detailed),  and  whether  or  not  the 
operator  has  previously  seen  the  message.  The  subjective  weight,  or 
utility,  that  the  user  places  on  each  attribute  is  estimated  on-line,  by 
an  adaptive  technique,  as  the  user  manually  selects  information.  The 
utilities,  in  combination  with  the  measured  attribute  levels,  permit 
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calculation  of  a  multi -attribute  utility  (MAU)  value  for  each  available 
message.  The  selector  mechanism  then  passes  to  the  operator  the 
message(s)  in  the  queue  with  the  highest  MAU.  In  situations  where 
several  messages  are  sent  to  an  operator  simultaneously,  the  model-based 
selector  can  reduce  the  size  of  the  transmitted  message  set  by  retaining 
only  those  messages  which  contribute  more  than  some  criterion  value  of 
utility.  For  example,  a  pruning  rule  has  been  demonstrated  which  first 
ranked  messages  in  order  of  decreasing  MAU;  then,  beginning  with  the 
message  with  the  highest  MAU,  each  successive  message  was  added  to  the 
set  only  if  its  individual  MAU  value  exceeded  15%  of  the  total  MAU  of  all 
messages  already  in  the  set  (Samet,  Weltman,  and  Davis,  1976). 

The  MAU  for  a  given  message  (i)  is  computed  on  the  basis  of  its 
level  on  each  attribute  and  the  current  importance  or  utility 

weight  for  that  attribute  (U.).  Thus 

MAUi  = 

The  A.,  are  derived  directly  from  characteristics  of  the  message,  and 

*  V 

these  levels  are  stored  in  the  data  base  together  with  the  text  of  the 
message.  The  U.  are  continuously  updated  in  accordance  with  a  weight 

V 

training  algorithm  applied  to  repeated  paired-comparison  preference 
choices  made  by  the  operator.  He  is  presented  with  the  headers  (i.e., 
vector  of  attribute  levels)  for  each  of  two  potentially  available 
messages  and  is  asked  to  state  a  preference  for  actually  receiving  one 
message  or  the  other.  As  the  operator  performs  his  task,  the  on-line 
weight  estimator  observes  his  choices  between  each  pair  of  message 
headers,  and  views  his  decision-making  as  a  process  of  classifying 
patterns  of  information  attributes.  The  estimator  predicts  each  choice 
by  classifying  the  attribute  patterns  by  means  of  a  linear  evaluation 
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(discriminant)  function.  These  predictions  are  compared  with  the 
operator's  actual  choices.  Whenever  they  are  incorrect,  the  adaptive 
error-correction  training  algorithm  shown  in  Table  2-1  is  used  to  adjust 
each  entry  (U.)  in  the  utility  vector.  More  details  concerning  this 
technique  can  be  found  in  Freedy,  Davis,  Steeb,  Samet  and  Gardiner  (1976) 
and  Samet,  et.  al .  (1976). 

2.2  Information  Pacing  Model 

The  pacing  model,  designed  to  present  one  message  at  a  time,  is 
based  on  the  implementation  of  a  secondary  task  (involving  choice 
reaction  time)  to  dynamically  assess  the  operator's  load  on  the  primary 
C3  information  processing  task  (i.e.,  the  reading  and  analysis  of 
messages).  An  adaptive  algorithm  continually  adjusts  the  pacing  rate  in 
accordance  with  fluctuations  in  assessed  operator  load.  In  addition,  the 
display  time  for  each  specific  message  is  adjusted  as  a  function  of  a  set 
of  predetermined  pacing  attributes.  The  parameter  values  required  by 
the  model  are  determined  in  an  operator  calibration  session.  A  more 
detailed  description  of  the  model  follows. 

A  single  pacing  algorithm  is  used  to  compute  the  display  time 
for  each  message  before  it  is  presented  on  the  screen.  The  algorithm 
has  two  basic  components.  The  first  component  involves  a  baseline 
display  time  which  is  adaptively  adjusted  in  accordance  with  gradual 
changes  in  operator  response  to  task  demands  as  measured  by  secondary 
task  performance.  The  adjustment  process  is  accomplished  reiteratively 
across  a  moving  window  of  task  performance  lasting  about  30  seconds. 

The  second  component  is  a  linear  function  of  specific  features  of  the 
particular  message  to  be  displayed. 


TABLE  2-1 
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Weight  Factor  Message  0) 


DIFFERENCE 


Attribute  Level 
in  Predicted 
Message  (k) 


The  two  components  are  combined  multiplicatively  to  determine  the 
display  time  (M.  )  for  message  i  during  interval  (i.e.,  window)  e: 

I  >e 

«.,,=  {!  *  D,)  . 


where 

Dj  =  b,p,i  +  bjPfj  +  ...  +  c 

and 


Tg  =  baseline  display  time  during  interval  e. 

D^.  is  the  estimated  relative  signed-deviation  in  display  time  for 
message  i  as  a  function  of  pacing-attribute  levels  (p^-].  P^2»  •••). 
regression  coefficients  (b^,  b^,  ...)»  and  a  constant  (c).  The 
parameters  of  the  equation  are  determined  from  an  analysis  of  operator 
performance  during  a  calibration  session  in  which  messages  are  self-paced. 
By  considering  a  large  number  of  these  self-paced  messages,  a  step-wise 
linear  regression  analysis  can  be  performed  with  the  levels  of  all 
attributes  of  message  i  as  the  independent  (predictor)  variables,  and 
the  relative  signed  deviation  of  the  observed  message  display  time  (M^) 
from  some  overall  mean  display  time  (M),  namely  D.j  =  (M^  -  M)/M,  as  the 
dependent  (criterion)  variable.  In  this  manner,  a  subset  of  message 
attributes  can  be  identified  to  serve  as  the  pacing  attributes 
(Pil,  Pt2»  ...).  Thus,  the  role  of  the  1  +  term  is  to  proportionately 
increase  or  decrease  the  current  baseline  display  time  in  terms  of  the 
expected  processing  time  requirements  for  the  particular  message  to  be 
presented. 

The  baseline  display  time  (T^)  for  interval  e  is  based  on  an 
adjustment  to  the  operator's  baseline  display  time  during  the  previous 


interval  (Tg_i)  in  accordance  with  his  level  of  primary  task  load  as 
inferred  from  his  observed  level  of  secondary  task  performance  during 
that  interval.  Performance  on  the  secondary  task  is  a  function  of  the 
operator's  reaction  time  on  a  decision  choice  task.  At  the  start  of  a 
system  paced  session,  the  baseline  display  time  is  initialized  at  the 
previously  calibrated  average  message  processing  time  for  the  operator 
(Tq).  Thereafter,  the  baseline  display  time  is  adjusted  on  an  interval' 
by-interval  basis  by  the  following  equation: 


where 


"^e  ~  ^e-1  e-r^X,  e-l^' 


>1  o  1  "  observed  level  of  secondary  task  performance  during 

L  9  C“  I 

interval  e-1; 


'X,  e-1 


Expected  level  of  secondary  task  performance  during 
interval  e-1,  determined  in  accordance  with  previously 
calibrated  operator  task  performance; 


A  =  Adaptive  increment  (i.e.,  sensitivity  coefficient 

determining  the  amount  of  change  in  T  per  unit 

difference  between  S,  ,  and  Sw  ,). 

L  9  S'*  I  A  9  S““  I 


The  expected  level  of  secondary  task  performance  is  computed  as 

a  function  of  S„,  T„,  and  T  ,,  where  S  is  the  standard  or  average  level 
0  0  e-1  0 

of  secondary  task  performance  achieved  by  the  operator  during  the 
calibration  session  for  which  his  average  message  processing  time  was  T^. 
The  equation  describing  this  computation  is: 


The  functional  block  diagram  in  Figure  2-2  shows  the  major 
components  of  the  information  selection  and  pacing  system.  These 
components  can  perhaps  be  most  easily  described  n  terms  of  their 
impact  upon  each  other. 

The  scenario-generated  "Message  Universe"  leaks  messages  into  an 
"Available  Ranked  Messages"  store  in  accordance  with  a  preselected  time- 
dependent  distribution  (e.g.,  Poisson).  This  process  depicts  the 
situation  in  the  real  world  where  messages  generated  from  the  external 
environment  are  not  available  for  display  until  they  have  been  stored 
in  a  computer  data  base.  In  addition,  the  time-tagged  message  inflow 
could  allow  for  message  age  (from  time  of  availability)  to  be  computed 
and  used  as  a  message  attibute  likely  to  affect  operator  information 
preferences. 

The  messages  in  the  available  store  are  continually  maintained 
in  rank  order  according  to  their  aggregate  MAU  value.  The  computation 
of  MAU  is  determined  by  the  "MAU  Model"  whose  input  includes  the 
respective  attribute  levels  determined  for  each  message,  and  the  attribute 
importance  weights  currently  assessed  for  the  operator.  Thus,  the 
available  information  store  represents  a  queue  of  messages  whose  MAU 
ranks  are  updated  continuously,  i.e.,  whenever  a  new  message  enters  the 
stack,  whenever  any  message  attribute  level  changes,  or  whenever  the 
operator's  attribute  importance  weights  change. 
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FIGURE  2-2.  INFORMATION  SELECTION  AND  PACING  MODELS 
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Messages  are  extracted  from  the  available  message  store  and 
displayed  to  the  operator  by  the  "Message  Selector"  and  "Message  Pacer". 
The  selector  simply  chooses  the  available  message  with  the  highest 
current  MAU  value.  The  pacer  takes  advantage  of  the  "Pacing  Algorithm" 
which  is  directly  dependent  upon  multiple  components.  Initially,  a 
"Baseline  Pacing  Rate"  for  the  operator  is  determined  from  previously 
collected  calibration  data.  This  baseline  rate  is  then  adjusted  by  the 
algorithm  in  two  separate  ways:  first,  a  message-by-message  adjustment 
is  made  by  the  "Message  Time  Calibrator"  unit.  This  calibration 
utilizes  the  level  of  specific  "Pacing  Attributes"  of  each  particular 
message  to  predict  how  much  its  display  time  should  be  increased  or 
decreased  with  respect  to  some  average  message  display  time.  Second, 
the  "Pacing  Algorithm"  adjusts  the  baseline  display  time  within  a  given 
time  interval  as  a  function  of  operator  load  during  the  previous 
interval.  The  adjustment  is  carried  out  by  the  "Load  Determinator", 
which  measures  load  on  the  primary  task  by  comparing,  across  the  previous 
interval,  the  operator's  observed  level  of  "Secondary  Task  Performance" 
IsjT^^h  arT^pected  level  (Sj^)  derived  from  a  "Standard  Secondary  Task 
Performance"  score.  The  expected  score  is  determined  from  parameters 
observed  to  be  appropriate  to  the  given  operator  in  a  calibration 
session.  Thus,  "Operator  Calibration  Data"  provide  an  important 
contributory  component  to  the  pacing  procedure. 

The  "Operator"  performs  the  primary  task  and  secondary  task 
concurrently;  his  major  goal  is  to  process  information  contained  in 
the  messages  in  order  to  provide  effective  "Primary  Task  Performance". 

The  successive  messages  (i.e.,  input  for  primary  task)  and  message 
headers  (i.e.,  input  -^or  secondary  task)  presented  to  the  operator  on 
his  "Display"  both  originate  from  the  "Available  Ranked  Messages" 
store.  The  message  headers  are  produced  by  the  "Secondary  Task 
Generator",  which  displays  the  headers  corresponding  to  two  messages 
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randomly  picked  from  the  store.  Finally,  the  operator's  decisions  or 
choices  among  headers  which  result  during  "Secondary  Task  Performance 
play  a  role  in  updating  the  "MAD  Model". 


2.4  Summary 


The  integrated  configuration  of  the  information  selection  and 
pacing  models  determine  what  new  information  should  be  supplied  to  the 
operator  and  when  it  should  be  supplied.  In  fact,  the  two  models  are 
logically  interconnected  through  the  secondary  task,  which  involves  the 
paired-comparison  choices  between  headers  of  available  information 
messages.  The  rate  of  decision  performance  on  this  task  is  pivotal  to 
the  pacing  model,  while  at  the  same  time,  the  actual  choices  serve  the 
important  additional  function  of  training  the  adaptive  information- 
selection  model  and  keeping  it  tuned  to  momentary  changes  in  information 
preferences. 


3.  EXPERIMENTAL  DEMONSTRATION 

3.1  Task  Scenario  and  Procedure 


The  information  selection  and  pacing  models  were  implemented^ 
into  a  C3  scenario  based  on  the  Tactical  and  Negotiations  Game  (TNG) 
(Streufert,  Castore,  and  Kilger,  1967).  This  scenario  was  selected 
because  it  possesses  considerable  task  complexity  and  provides  results 
with  applications  to  real-world  decision  making.  The  scenario  required 
the  operator  to  process  information  messages  and  make  situation 
assessments  concerning  the  military,  intelligence,  negotiation,  and 
economic  activity  of  a  small,  underdeveloped,  fictitious  nation  called 
Shamba,  which  is  plagued  by  an  internal  revolution  and  foreign 
intervention.  The  four  activity  areas  were  matched  in  scope  and 
complexity,  and  each  was  presented  to  the  operator  in  a  separate 
experimental  session.  The  object  of  the  task  was  to  learn  the  message 
content  and  to  correctly  diagnose  the  strategy  apparently  being  followed 
by  an  organized  rebel  movement. 

Corresponding  to  each  of  the  four  areas  of  activity,  30  information 
messages  were  available  in  the  data  base.  This  set  was  composed  of  15 
pairs  of  messages,  with  each  pair  consisting  of  a  detailed  and  a  summary 
version  of  the  content.  An  example  of  the  text  of  a  detailed  message 
within  the  intelligence  area  is  provided  in  Figure  3-1.  The  summary 
version  of  the  message  pair  is  as  follows:  "Agent  O.S.  heard  that  the 
enemy  may  be  planning  large  attacks  on  a  major  transportation  route  in 
central  Shamba." 


The  experimental  hardware  consisted  of  a  20-row  x  80-column  Beehive-100 
cathode  ray  tube  (CRT)  display,  on-line  to  a  PDP-11/45  computer  with  a 
Unix  operating  system. 
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NUMBER 


AREA 


SPECIFICITY 


STATUS 


14  Intelligence  Detailed  Unseen 


AGENT  J.S.  HEARD  DISCUSSIONS  INDICATING.  PLANS  FOR  A  MAJOR  ENEMY 
OFFENSIVE  SOMEWHERE  ALONG  THE  RAILROAD  BETWEEN  MCKOSAM  AND  SAVIN 
HE  WAS  NOT  SURE  WHERE  ENEMY  STAGING  AREA  FOR  THIS  OPERATION  IS, 
BUT  THE  EQUIPMENT  MOVING  ACROSS  THE  ONDULU  RIVER  RECENTLY 
INDICATES  IT  MIGHT  BE  THE  SWAMS  IN  SECTOR  0-6. 


[  jcHOICE  AREA  SPECIFICITY  STATUS 


Seen 

Unseen 


1  Intelligence  Detailed 

2  Intelligence  Summary 


The  messages  were  displayed  as  shown  in  Figure  3-1;  one  message 
was  shown  at  a  time,  yet  any  message  could  be  presented  more  than  once. 
When  messages  were  automatically  paced,  remaining  display  time  in  seconds 
was  shown  along  side  the  message;  this  number  was  decremented  every  10 
seconds,  but  when  less  than  10  seconds  were  left,  the  countdown  was  every 
second.  Above  the  message  text,  header  information  appeared  which 
included  message  identification  number,  content  irea,  specificity,  and 
status.  Only  the  latter  two  items  were  employed  as  message  attributes 
within  the  experimental  demonstration  described  here.  The  speci ficity 
attribute  took  on  levels  of  0  (summary  message)  and  1  (detailed  message). 
Status  referred  to  the  number  of  times  that  the  message  was  previously 
seen  by  the  operator.  For  simplicity,  when  status  =  0,  "unseen"  appeared 
on  the  display;  when  status  =  1,  "seen"  appeared;  when  status  =  2, 

"seen  2"  appeared;  etc.  For  a  given  message,  specificity  determined 
a  fixed  attribute  level,  whereas  the  level  for  status  was  dynamically 
updated  as  a  function  of  operator  viewing. 

The  secondary  task  involved  a  preference  choice  between  two 
potentially  available  messages  randomly  selected  from  the  data  base 
(i.e,,  "Available  Ranked  Messages"  store).  For  example.  Figure  3-1 
presents  a  choice  between  a  detailed  intelligence  message  that  the 
operator  has  already  seen  (choice  1)  and  a  summary  intelligence  message 
that  he  has  not  seen  (choice  2).  A  new  secondary  task  display  came  on 
and  off  the  screen  simultaneously  as  the  operator  proceeded  to  read 
messages;  the  on  time  was  6  seconds,  and  the  off  time  varied  at  random 
within  an  interval  from  5  to  11  seconds.  The  operator  indicated  his 
message  preference  by  typing  either  a  1  or  2  into  the  keyboard,  and  his 


2 

In  other  demonstrations  of  the  system,  message  content  area  has  been 
successfully  manipulated  as  an  attribute  in  a  modified  version  of  the 
MAU  model . 
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choice  reaction  time  was  used  as  a  measure  of  secondary  task  performance. 
If  he  failed  to  respond  during  the  period  when  the  choice  was  displayed, 
his  reaction  time  was  recorded  as  6  seconds.  The  operator  was  instructed 
on  the  importance  of  the  secondary  task,  and  that  he  would  not  immediately 
receive  the  particular  messages  he  chose  but  that  the  system  would  keep 
track  of  his  preferences  and  would  soon  select  messages  for  him 
accordingly.  In  fact,  each  time  the  operator  responded,  his  choice  was 
predicted  by  the  MAU  model,  and  his  utility  weights  for  the  message 
attributes  were  adjusted  given  an  incorrect  prediction. 

The  experimental  demonstration  was  conducted  with  six  college 
students  serving  as  operators.  Each  operator  was  run  individually  and 
performed  the  same  basic  tasks.  After  reading  a  manual  providing 
background  on  the  TNG  scenario,  and  receiving  comprehensive  instructions 
concerning  the  primary  and  secondary  tasks  and  on  how  to  interact  with 
the  experimental  console,  operators  completed  a  set  of  problem  sessions. 
The  initial  session  was  a  practice  session  in  which  the  pacing  of  messages 
was  under  the  control  of  the  operator,  i.e.,  self-pacing.  The  remaining 
three  sessions  were  experimental  sessions  in  which  message  pacing  was 
controlled  by  either  the  operator  or  the  computer;  the  specific  pacing 
manipulations  applied  in  each  of  the  three  sessions  are  described  in 
paragraphs  below. 

The  problem-solving  procedure  was  identical  throughout  all 
sessions.  Prior  to  receiving  messages  about  the  one  activity  area 
presented  in  the  session  (military,  intelligence,  economic,  or 
negotiations),  the  operator  was  given  four  alternative  hypotheses 
concerning  the  rebel  movement's  strategy,  only  one  of  which  was  actually 
correct.  A  session  lasted  exactly  six  minutes,  during  which  time  the 
operator  read  messages  and  performed  the  secondary  task  simultaneously. 

At  the  conclusion  of  this  period,  the  operator  was  required  to  provide 


a  probability  vector  over  the  four  strategy  alternatives,  with  each 
probability  reflecting  his  confidence  that  the  respective  alternative 
was  correct.  In  addition,  a  true-false  and  a  multiple-choice  quiz  were 
administered  to  assess  the  operator's  knowledge/memory  of  message  content. 
These  tests  each  contained  15  different  questions,  with  one  question 
corresponding  to  each  message  pair  available  within  the  data  base.  Thus, 
for  a  given  operator  and  session,  two  measures  of  information  processing 
performance  were  available,  namely,  probability  on  the  correct  strategy 
(y)  and  number  of  correct  answers  (z)  for  the  30  objective  questions. 

These  measures  were  used  in  conjunction  with  an  index  of  secondary  task 
performance  [number  of  missed  selections  (w)]  to  compute  a  bonus  payoff 
for  each  operator  which,  when  added  to  his  flat-rate  pay  ($10  for  2?^ 
hours),  resulted  in  an  average  overall  earnings  of  about  $20. 

For  all  operators,  the  military  activity  area  was  presented  in  the 
practice  session.  At  the  conclusion  of  this  session,  each  operator  received 
feedback  concerning  his  performance.  He  then  proceeded  to  complete 
sessions  1,  2,  and  3.  The  activity  areas  (intelligence,  economics, 
negotiations)  were  assigned  to  sessions  according  to  a  Latin  Square  design 
so  that  each  group  of  two  operators  received  a  different  assignment 
(ordering).  Feedback  on  performance  during  the  three  experimental  sessions 
was  not  given  until  the  last  session  was  completed.  A  description  of  the 
pacing  manipulations  during  each  session,  and  the  type  of  corresponding 
calibration  data  provided,  are  described  below,  these  manipulations  refer 
to  message  pacing  only,  since  the  secondary  task  was  automatically  paced 
in  all  cases. 

2 

The  bonus  payoff  in  dollars  (BP)  for  performance  on  a  given  session  was 
determined  by  th"  ‘following  algorithm;  BP=[2  +  log  (y  +  .01)]  + 

[.lOz]  -  [.lOw]. 


Session  1:  Self-Pacing  (SP).  In  this  mode  of  system  operation, 
the  operator  reviewed  each  message  for  as  long  as  he  wanted.  When  ready 
to  proceed,  he  pressed  a  button  on  the  CRT  keyboard,  and  the  next  message 
was  immediately  displayed.  Results  from  this  session  provided  an  average 
self-pacing  message  rate  (M)  for  the  operator  as  well  as  input  which 
could  be  used  to  determine  the  operator-suited  regression  coefficients 
for  the  component  of  the  pacing  algorithm. 

Session  2:  Message-Based  Pacing  (MP).  During  this  session, 
messages  were  automatically  paced  by  the  computer.  The  baseline  pacing 
rate  was  set  for  the  operator  at  a  fixed  value,  namely,  T^  =  M,  and  the 
display  time  (M^ )  for  a  given  message  (i)  was  computed  by  =  (1+D^)T^. 
Thus,  in  this  session,  message  display  time  fluctuated  around  the 
operator's  self-paced  message  processing  rate  (M)  as  a  function  of  D., 
i.e.,  in  accordance  with  the  levels  of  the  pacing  attributes  (p)  for 

4 

message  i.  As  it  turned  out,  the  same  regression  equation  was  employed 
to  compute  D^.  for  all  operators,  namely: 

D.  =  .6P^  +  .7P2  -  .7, 

where 


P.|  =  number  of  lines  of  message  text  appearing  on  CRT; 

P2  =  1  for  an  unseen  message, 

0  for  a  message  scene  more  than  once. 


4 

This  equation  accounted  for  about  50%  of  the  variance  in  the  prediction 
of  D^.. 
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Thus,  for  example,  if  T  =  20  seconds,  then  a  two-line,  already-seen 
message  was  displayed  for  10  seconds,  and  a  four-line,  not-yet-seen 
message  was  shown  for  28  seconds. 

In  the  MP  session,  to  summarize,  pacing  was  a  function  of  message 
parameters  and  a  pre-cal ibrated  operator  parameter  (M);  however,  a  dynamic 
measure  of  operator  load  as  assessed  on-line  was  not  incorporated  into  the 
pacing  procedure.  From  operator  performance  in  this  session,  the 
calibration  parameter  (i.e.,  standard  level  of  secondary  task 
performance)  was  determined,  and  its  ratio  to  was  used  to  adjust 
pacing  according  to  assessed  operator  load  in  the  subsequent  session. 

Session  3:  Load-Based  and  Message-Based  Pacing  (LMP).  In  this 
final  session,  messages  were  again  automatically  paced  by  the  computer; 
however,  this  time  the  full  capability  of  the  pacing  system  was  applied. 

In  addition  to  the  adjustment  of  display  time  as  a  function  of  message 
pacing  attributes  as  done  in  the  previous  session  (MP),  the  current  pacing 
manipulation  also  took  advantage  of  the  on-line  assessment  of  operator 
load.  In  this  session,  therefore,  the  baseline  display  time  (T^)  was 
initialized  at  a  value  of  T^,  and  was  then  successively  adjusted  by  the 
adaptive  equation  according  to  the  difference  between  the  observed  (S^) 
and  expected  (S^^)  levels  of  secondary  task  performance  during  each 
30  second  interval.  Then,  to  obtain  the  display  time  (M.  )  for 
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message  i  during  interval  e,  the  current  value  of  T^  was  multiplied  by 
the  message  adjustment  factor  (1  +  D^. ). 

3.2  Demonstration  Results 


The  experimental  demonstration  described  in  the  preceding  section 
was  not  intended  to  assess  the  relative  effectiveness  of  computer- 
controlled  information  selection  and  pacing.  Instead,  the  goal  was  to 


m  ^ 


demonstrate  that  these  functions,  normally  controlled  by  the  human 
operator,  could  be  automated  via  a  user-based  model  so  that  the 
information  integration  achieved  by  the  operator  can  be  maintained  at 
an  acceptable  level.  Because  of  the  requirement  of  operator  calibration, 
therefore,  no  attempt  was  made  to  control  for  the  effect  of  session  order 
(1=SP,  2-MP,  3-LMP)  on  operator  performance.  Since  the  effects  of  these 
session  manipulations  were  confounded  with  any  possible  effects  due  to 
learning  or  task  familiarity,  data  obtained  in  these  experimental  runs 
were  not  subjected  to  tests  of  statistical  significance. 

Performance  data  were  averaged  across  the  six  operators,  and  the 
means  for  selected  dependent  measures  are  presented  in  Table  3-1.  Without 
making  statistical  inferences  concerning  the  results,  the  following 
observations  can  be  noted.  The  small  differences  among  sessions  with 
respect  to  "number  of  message  presentations"  and  "message  display  time" 
do  not  signify  any  important  performance  differences;  rather,  they  reflect 
slight  variations  in  the  types  of  messages  displayed  (i.e.,  pacing 
attribute  vectors  and  their  impact  on  D^-)  and  rounding  errors  within  the 
pacing  algorithm.  "Secondary-task  reaction  time"  was  faster  in  the  MP 
and  IMP  sessions  than  in  the  SP  session,  possibly  suggesting  that  when 
operators  are  relieved  of  deciding  when  to  proceed  to  the  next  message, 
their  residual  task  capacity  can  increase.  Had  the  level  of  secondary 
task  performance  been  much  slower  or  faster  in  the  IMP  session  than  in 
the  MP  session,  then  the  pacing  rates  in  the  two  sessions  would  have 
differed  respectively;  but  this  phenomenon  did  not  occur.  As  for 
information  processing  performance,  the  results  for  "percentage  of 
operators  with  highest  probability  on  the  correct  strategy"  and  "percent 
correct  answers"  to  test  questions  indicated  that  operators  did  a  little 
better  in  the  MP  and  IMP  sessions  than  in  the  SP  session. 


TABLE  3-1.  SUMMARY  OF  AVERAGE  RESULTS 


COMPUTER-PACED  SESSIONS 


PERFORMANCE 

INDEX 


SELF-PACED  LOAD-AND 

(SP)  SESSION  MESSAGE-PACED  (MP)  MESSAGE-PACED  (LMP) 


No.  of  Message 

Presentations  25.3  27.0  29.3 


Message  Display 

Time  (sec.)  14.3  13.9  14.0 


Secondary  Task 
Reaction  Time 
(sec. ) 


3.3 


2.7 


2.7 


No.  of  Operators 
with  Highest 
Probability  on 

Correct  Strategy  45  5 


%  Correct  Answers 


60.0 


71.0 


64.3 


In  general,  the  results  do  not  indicate  any  very  large  differences 
in  information  processing  performance  as  a  function  of  whether  messages 
are  paced  manually  by  the  operator  or  paced  automatically  by  the  computer. 
This  finding  agrees  with  empirical  results  obtained  by  Levine,  Samet,  and 
Brahlek  (1974)  that  it  made  little  difference  whether  information  was 
presented  automatically  or  upon  request,  as  long  as  the  rate  of 
information  presentation  was  approximately  equivalent.  Furthermore,  the 
data  suggest  that  accurate  model -based  prediction  of  required  display 
time  on  a  message-by-message  basis  may  be  a  reasonable  approach  to  the 
control  of  pacing,  without  the  necessity  of  assessing  operator  load 
on-line  (by  an  obtrusive  secondary  task  or  other  method)  and  including 
this  measure  as  an  additional  adjustment  within  the  pacing  mechanism. 
However,  when  interpreting  this  suggestion,  it  should  be  kept  in  mind 
that  the  message-based  pacing  attributes  used  here  (number  of  textual 
lines  and  whether  the  message  was  previously  viewed)  did,  in  fact, 
reflect  upon  immediate  operator  load. 


4.  DISCUSSION 


Two  aspects  of  this  research  have  been  selected  for  discussion. 
First,  a  few  advantages  of  the  multi-attribute  modeling  approach  are 
mentioned.  Second,  an  ongoing  research  program  is  outlined  which  is 
intended  to  extend  the  applications  of  these  models  to  the  computer 
control  of  information  flow  among  a  cooperative  group  of  operators. 

4.1  Advantages  of  the  Multi -Attribute  Approach 

The  adaptive  models  for  information  selection  and  pacing  developed 
in  this  research  are  characterized  by  several  attractive  features.  These 
features,  briefly  described  below,  can  be  seen  as  advantages  which  endorse 
the  application  potential  of  the  approach.  Although  the  advantages  arise 
out  of  the  theoretical  structure  of  the  models,  especially  the 
decomposition  property,  they  have  all  been  empirically  illustrated  to 
some  degree  in  the  experimental  demonstration. 

The  models  are  considerably  general .  They  can  be  applied  in  a 
variety  of  situations  where  information  messages  can  be  decomposed  into 
a  small  set  of  manageable,  quantifiable  attributes.  These  attributes 
must  be  logically  taken  into  account  by  the  operator  in  judging  the 
utility  of  potentially  available  information.  That  is,  they  must 
directly  impact  upon  his  choices  among  competing  information  messages. 
Several  decision  making  environments  have  already  been  demonstrated  to 
fit  this  paradigm  (e.g.,  Hayes,  1964;  McKendry,  Enderwick,  and  Harrison, 
1971;  Samet,  1975). 

The  models  are  inherently  flexible.  If  accuracy  of  prediction 
of  information  selection  or  pacing  behavior  is  not  sufficient  (i.e.,  if 
selection  or  pacing  attribute-weights  cannot  be  appropriately  adjusted). 


additional  features  or  attributes  can  iteratively  be  added  and  irrelevant 
ones  deleted.  The  response  to  dynamic  changes  in  conditions  is  similarly 
flexible;  in  instances  where  conditions  change  rapidly  and  radically, 
new  sets  of  weights  trained  for  the  conditions  can  be  substituted. 

The  models  are  parsimonious.  They  need  only  assess  an  operator's 
selection  or  pacing  weights  for  a  limited  number  of  information  dimensions 
or  attributes.  Besides  significantly  minimizing  the  models'  computational 
needs  and  software  complexity,  this  feature  is  in  consort  with  the  result 
of  psychological  experiments  (e.g.,  Hayes,  1964;  Wright,  1974)  and 
contemporary  decision  theory  (e.g.,  Hogarth,  1975;  Tversky  and  Kahneman, 
1974);  namely,  that  a  decision  maker  can  perform  an  intuitive  conscious 
weighting  and  aggregation  of  only  a  relatively  small  number  of  what  he 
considers  to  be  the  important  dimensions  common  to  the  decision 
eval uation. 

The  models  are  robust.  Like  other  linear  composition  models,  their 
performance  (i.e.,  capability  to  mimic  the  information  processing  behavior 
of  an  operator)  is  not  significantly  degraded  by  proportionately  small 
perturbations  in  the  model's  parameters  (Dawes  and  Corrigan,  1974). 

4.2  Further  System  Develooraent 


Research  and  development  efforts  are  continuing  to  provide  a 
far-reaching  extension  of  the  adaptive  modeling  technique  for  information 
selection  from  the  domain  of  the  individual  to  that  of  a  group.  The 
fundamental  idea  is  to  route  irformat’cn  to  individual  operators  in 
accordance  with  a  group-based  model  of  information  preferences  and  needs. 
This  multi-man  supervisory  system  takes  into  account  each  operator's 
preferences  both  for  receiving  and  for  sending  information  messages,  as 
well  as  his  position  (i.e.,  relative  power)  or  communicative  role  within 


the  group.  Basically,  the  pattern  of  information  distribution  is 
expressed  as  a  compound  set  of  multi-attribute  information  utility 
models  which  comprise  each  operator's  information  preferences  for 
himself  and  for  each  of  the  other  group  measures. 

These  individual-based  models  are  continuously  integrated  by  the 
system  so  that  the  combined  roles  of  the  individual  group  members  (i.e., 
pattern  of  information  exchange),  commonly  referred  to  as  the  group 
"structure"  or  "communication  network",  can  be  dynamically  altered  in 
real-time.  Essentially,  the  idea  is  to  adaptively  distribute  "power" 
or  control  over  information  flow  among  group  members  in  accordance  with 
measures  of  each  operator's  relative  expertise.  The  goal  of  these 
manipulations  is  to  create  the  most  effective  policy  of  information 
exchange  in  response  to  situational  needs. 

In  the  system  under  development,  a  computer  program  continuously 
monitors  the  "goodness"  or  relevance  of  messages  received  by  each  group 
member,  and  accordingly,  it  automatically  adjusts  group  structure  in 
real  time  in  the  direction  which  offers  more  effective  message 
communication  among  group  members.  Furthermore,  the  allocation  of 
power  over  information  flow  can  take  on  a  different  distribution  (i.e., 
group  structure)  vis  a  vis  each  individual  in  the  network.  Structure 
is  manipulated  by  means  of  the  relevant  knowledge-base  and  information- 
distribution  power  demonstrated  by  each  group  member,  and  power  is 
considered  to  be  proportional  to  an  operator's  expertise  to  both  evaluate 
and  route  information.  This  approach  fits  the  theoretical  framework  of 
Wood  (1973),  which  conceptualizes  the  group  decision  process  as  "a 
multi-phased  process,  in  which  participation,  multiple  bases  of  power, 
and  interaction  dynamics  affect  power  relationships".  In  this  view,  the 
distribution  of  power  is  sensitive  to  both  situational  and  individual 
differences,  and  it  serves  as  an  intervening  variable  which  determines 
group  structure. 
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The  notion  of  having  a  computer,  rather  than  people,  manipulate 
group  structure  is  particularly  appealing  in  light  of  a  recent  hypothesis 
stated  and  confirmed  by  Bourgeois,  McAllister,  and  Mitchell  (1977), 
namely:  "...environment-organization  contingency  theories  are  not  only 
counter-intuitive  but  in  fact  require  organizational  participants  to 
respond  in  a  manner  quite  opposite  to  their  natural  inclinations".  Thus, 
for  example,  a  member  of  a  decision  making  team  might  be  intuitively 
inclined  to  decrease  information  communication  at  a  time  when  he  should 
in  fact  increase  it.  In  such  cases  of  consistently  poor  human  performance 
the  normative  rule-based  procedures  of  computer  control  could  serve  as 
a  potentially  valuable  information  processing  aid. 

An  interesting  and  important  characteristic  of  this  computer- 
controlled  model  for  information  exchange  is  that  an  operator  (i.e., 
receiver  of  information)  need  not  be  informed  of  the  current  status  of 
the  communication  network,  i.e.,  how  much  relative  information-control 
power  he  or  anyone  else  has.  The  implication  of  anonymity  is  that  non¬ 
constructive  bias,  resulting  from  the  social  psychology  of  role 
relationships  in  a  people-controlled  communication  network,  can  be  removed 
from  the  information-distribution  system.  Such  bias  is  represented,  for 
example,  in  the  common  and  sometimes  tragic  problem  brought  on  by  status 
differential  when  a  subordinate  fails  to  tell  his  commander  something 
that  the  commander  would  ordinarily  need  or  would  want  to  know  in  a 
particular  situation.  In  the  system  under  development,  such  information 
would  automatically  be  passed  to  the  commander  by  a  consistent, 
intelligent  model  which  incorporates  his  stable  preferences  for  receiving 
specific  information,  and  which  is  not  affected  by  the  psychological 
"noise"  in  the  communication  channels. 
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Conclusion 


The  work  described  here  allowed  the  development  and  demonstration 
of  a  computer-based  adaptive  system  which  automatically  selects  and  paces 
information  messages  for  a  human  operator.  It  is  anticipated  that  these 
kinds  of  user-based  models  can  be  incorporated  into  an  integrated  complex 
of  intelligent  aids  for  supporting  real-time  management  of  information 
flow.  Considerable  empirical  work,  however,  will  be  required  to  assess 
the  impact  of  such  man-computer  systems  on  the  quality  of  information 
processing  and  decision  making. 
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